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Abstract 

Background: Maximizing efficiency in drug development is important for drug developers, policymakers, and 
human subjects. Limited funds and the ethical imperative of risk minimization demand that researchers maximize the 
knowledge gained per patient-subject enrolled. Yet, despite a common perception that the current system of drug 
development is beset by inefficiencies, there remain few approaches for systematically representing, analyzing, and 
communicating the efficiency and coordination of the research enterprise. In this paper, we present the first steps 
toward developing such an approach: a graph-theoretic tool for representing the Accumulating Evidence and 
Research Organization (AERO) across a translational trajectory. 

Methods: This initial version of the AERO model focuses on elucidating two dimensions of robustness: (1) the 
consistency of results among studies with an identical or similar outcome metric; and (2) the concordance of results 
among studies with qualitatively different outcome metrics. The visual structure of the model is a directed acyclic 
graph, designed to capture these two dimensions of robustness and their relationship to three basic questions that 
underlie the planning of a translational research program: What is the accumulating state of total evidence? What has 
been the translational trajectory? What studies should be done next? 

Results: We demonstrate the utility of the AERO model with an application to a case study involving the antibacterial 
agent, moxifloxacin, for the treatment of drug-susceptible tuberculosis. We then consider some possible elaborations 
for the AERO model and propose a number of ways in which the tool could be used to enhance the planning, 
reporting, and analysis of clinical trials. 

Conclusion: The AERO model provides an immediate visual representation of the number of studies done at any 
stage of research, depicting both the robustness of evidence and the relationship of each study to the larger 
translational trajectory. In so doing, it makes some of the invisible or inchoate properties of the research system 
explicit - helping to elucidate judgments about the accumulating state of evidence and supporting decision-making 
for future research. 
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Background 

Maximizing efficiency across a drug development trajec- 
tory is important for drug developers, policymakers, and 
human subjects. Limited resources and the ethical imper- 
ative of risk minimization demand both that researchers 
design and plan their studies to maximize the knowledge 
gained per patient-subject enrolled and that funders and 
ethical review boards hold them to this standard. 

Unfortunately, this standard is often not met. The infa- 
mous failure of torcetrapib at phase 3 illuminated inef- 
ficiencies in cardiovascular drug development [1-3]. In 
cancer, the cases of sunitinib for treating hepatic cancer 
and bevacizumab for treating gastric cancer were both 
examples of drugs that advanced into phase 3 testing with- 
out supporting phase 2 evidence [4]. Across the entire 
spectrum of drug development, nearly one-third of the 
drugs abandoned at phase 2 are considered failures not for 
lack of efficacy, but for strategic' reasons [4] . Even within a 
single company, as Pfizers internal review showed, almost 
half of their phase 2 proof-of-concept studies (43%) failed 
to test the target mechanism of action adequately [5] . 

These problems all contribute to the common per- 
ception that the current system of drug development is 
beset by inefficiencies [4,6,7], and the growing number 
of calls for greater coordination across the drug develop- 
ment enterprise [8-10]. Yet, there remain no systematic 
approaches for representing, analyzing, and communi- 
cating the efficiency and coordination of the research 
enterprise. The available techniques of systematic review 
and statistical meta-analysis can only tell a part of this 
story - elucidating partial cross sections of the evidence 
- but they do not offer any synthesis for the accumulating 
state of evidence across the entire translational trajectory. 
For example, a meta-analysis is well suited to identifying 
trends across a series of similar trials, but it is useless for 
understanding transitions between the different phases of 
research. How did the research program progress from 
pre-clinical to clinical studies? How did the results of 
phase 1 studies compare with phase 2? What does the 
failure of translation from pre-clinical to clinical tell us 
about what we ought to do next? These kinds of questions 
are entirely consistent with the aim of comprehensively 
understanding the translational research enterprise, but 
they are not questions for which a statistical meta-analysis 
is helpful. 

In this paper, we present a first step toward developing 
a broader, more comprehensive approach to represent- 
ing and understanding a translational research program: 
A graph-theoretic tool for representing the Accumulat- 
ing Evidence and Research Organization (AERO) within 
a translational trajectory. We begin in the next section 
by describing the basic methodology - the components 
of the AERO model and how these are used to con- 
struct an AERO graph. We then apply this tool to a case 



study involving the antibacterial agent, moxifloxacin, for 
the treatment of drug-susceptible tuberculosis. We close 
by discussing the potential utility of our approach for 
improving the organization, coordination, and efficiency 
of the drug development enterprise. 

Methods 

The conceptual foundations of the AERO model emerged 
out of work in the philosophy of science to develop a 
general methodology for representing robust scientific 
evidence [11]. Although there are many different dimen- 
sions of robustness that figure in drug development, the 
version of the AERO model we present here focuses on 
elucidating just two of these: (1) the consistency of results 
among studies with an identical or similar outcome met- 
ric; and (2) the concordance of results among studies with 
qualitatively different outcome metrics. For example, a 
series of animal experiments that all show a similar effect 
size for a new drug is evidence of consistency. A positive 
direction of effect on both the surrogate outcome used in 
a phase 2 trial and the clinical outcome used in a phase 3 
trial is evidence of concordance. 

The visual structure of the AERO model is a directed 
acyclic graph (DAG), designed to capture these two 
dimensions of robustness and their relationship to three 
basic questions that underlie the planning of a transla- 
tional research program: What is the accumulating state 
of total evidence? What has been the translational tra- 
jectory? What studies should be done next? We discuss 
each of these questions in turn, showing how they are 
represented in the model. 

Representing the accumulating state of total evidence 

The accumulating state of evidence in a drug develop- 
ment trajectory is constituted by discrete experiments, 
proceeding from the pre-clinical in vitro and in vivo exper- 
iments to the phase 1, 2 and 3 clinical trials. In the AERO 
model, we represent each experiment as a node (that is, 
a vertex) arranged in a two-dimensional space: an x-axis, 
representing time, and a y-axis, representing the phase of 
research. The nodes are then color-coded according to the 
direction of their outcome: studies in support of further 
research (for example, positive results) are green, studies 
against further research (for example, negative results) are 
red, and studies ambivalent toward further research (for 
example, inconclusive results) are yellow. 

Figure 1 is an example AERO graph for a research pro- 
gram with eight experiments across three phases in a 
five-year span. Although this graph is incomplete, since it 
does not yet include the arrows (that is, the edges) to illus- 
trate the translational trajectories, it nevertheless captures 
some features of the research program that are essential 
to understanding the state of total evidence. For exam- 
ple, we can see that the translation from animals into 
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Figure 1 Consistency of results. Studies are shown as vertices. The 
graph shows eight experiments across five years with some degree of 
consistency evident at every phase of research: Both of the in vitro 
studies, two of the three in vivo, and two of the three phase 1 studies 
were positive. There is also some inconsistency within in vivo Q3i ) and 
phase 1 (72); nevertheless, the accumulating state of evidence is 
largely positive and the transitions between phases appear relatively 
smooth. 



humans (that is, in vivo into phase 1) is relatively smooth. 
Two positive animal studies {fa, fa) suggest a potential 
for efficacy in humans and two positive phase 1 studies 
(yi> 73) suggest a well- tolerated drug. There is also evi- 
dence of consistency at each phase {a\ and 0^2, fa and fa, 
yi and 73). Given that consistency between experiments 
serves to verify findings and control for biases or ran- 
dom errors, which may distort the results of any single 
experiment, this is a desirable feature for the system to 
have. 

Figure 1 also shows some evidence of concordance. The 
overall trend of the experimental findings is largely posi- 
tive across each phase. With the exception of one negative 
in vivo study and one inconclusive phase 1 study, the 
results have all been favorable to further research. This 
degree of concordance is another desirable feature of the 
system. Depending upon the predictive power of the ani- 
mal model, this pattern of robust results provides good 
reason for thinking that the experimental agent may be 
efficacious and safe. 

Finally, assuming that fa is later than 012 and y\ is later 
than fa, we can see that there was a de facto threshold of 
two positive studies at each phase before proceeding to 
the next. These thresholds between phases are an impor- 
tant property of the translational trajectory, representing 
critical (and often expensive) decision points: When is the 
pre-clinical evidence sufficient to initiate human trials? 
When is the phase 2 evidence sufficient to justify a piv- 
otal phase 3 trial? Many translational failures (like those 
mentioned above) can be understood as the result of poor 
or inappropriate thresholds, wherein the evidence was not 



sufficiently mature to warrant advancing a candidate to a 
later phase of research. 

One plausible way to set phase thresholds is to require 
a certain number of positive studies at each phase before 
proceeding to the next. Indeed, the US FDAs licensure 
requirement for two positive phase 3 trials can be thought 
of as just such a threshold. However, two positives is not 
the only viable threshold. For example, a certain number 
of negative studies within a phase could require either that 
an agent be sent back to an earlier phase or abandoned 
entirely. It may also be reasonable to require that there is 
a sufficient number of both positive and negative studies 
at each phase - the positive studies supporting the effi- 
cacy of the therapeutic ensemble for clinical translation 
and the negative studies supporting a theoretical evidence 
base that directly informs clinicians about how the exper- 
imental agent should not be used [12]. Leaving aside the 
question of which phase threshold to use, either in gen- 
eral or for some specific research domain, it is sufficient 
for the purposes here to observe that the AERO graph has 
the virtue of rendering each of these thresholds visually 
explicit. 

Representing the translational trajectory 

In Figure 2, we expand on Figure 1, introducing three 
phase 2 studies as well as arrows between studies. The 
arrows represent more precisely the sequence of studies 



Phase Year 
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Figure 2 Concordance in a trajectory. Edges show the intellectual 
lineage. The graph shows eleven experiments across six years. The 
edges between studies represent the intellectual lineage between 
them and illustrate the translational research trajectories. Some 
trajectories are perfectly concordant (for example, 
ofi aj -> -> Y3 -> h), while others show discordance (for 
example, a\ -> ft] orai -> -> $2 -> Yi -> 8])- The phase 2 
studies are also highly inconsistent (that is, one positive, one negative, 
and one inconclusive study), indicating a relatively rough transition 
into this phase. Given that phase 2 was initiated after only one 
positive phase 1 trial, this may indicate that the threshold of evidence 
used to transition into phase 2 is too low and ought to require at least 
some degree of consistency. 
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and capture the intellectual lineage across the transla- 
tional trajectory. For example, a phase 2 study that uses 
the same dosage identified in a phase 1 study should be 
connected by an arrow leading out from the phase 1 node 
and into the phase 2 node (for example, y\ to 8\). Similarly, 
a phase 1 human pharmacokinetic and pharmacodynamic 
study should be connected to the prior in vivo study that 
identified the effective blood concentration (for example, 
ft to yi and 5/3). Borrowing terminology from the lan- 
guage of graph theory, we refer to studies downstream in 
the intellectual lineage as children,' and studies upstream 
as parents' (we will have more to say about how parentage 
is established, when we discuss the example in the next 
section). 

This additional structure enriches judgments about 
consistency and concordance. Where before we could 
only suggest concordance by the number or ratio of pos- 
itive results at each phase, now we can track a discrete 
translational trajectory. For example, the trajectory from 
a\ to «2 to to yi is a sequence of four positive studies 
across three phases. Each of these studies built on the evi- 
dence in the prior study, and each showed a concordant 
result. 

Notice, however, that the story of this trajectory gets 
more complicated when we follow it into phase 2. The 
first phase 2 study, 8\, was negative, despite following up 
the evidence from y\. But negative results are not neces- 
sarily uninformative results. Thus, as the figure shows, a 
later positive phase 1, 73, built upon 8\, and eventually led 
(in conjunction with ^3), to the positive phase 2 result in 
82. The complete story of this trajectory, while not one of 
perfect concordance or consistency, can nevertheless be 
described as favorably robust overall. 

Figure 2 also reveals that there was only positive phase 
1 study (71) before phase 2 research began (that is, 73 
is subsequent to 8\), Again, this represents a de facto 
(if not explicit) judgment about the minimum thresh- 
old of evidence necessary to justify proceeding to the 
next phase. Since consistency requires at least two exper- 
iments, advancing into the next phase of research on 
the basis of only one experiment means that this min- 
imum threshold included no evidence of consistency at 
the immediately preceding phase. For exactly the same 
reasons that consistency is desirable, failing to have con- 
sistency before proceeding to a subsequent phase is, in 
general, an undesirable feature of a research system: No 
verification of findings, no control for bias or random 
error. While a single, well-designed study may be suffi- 
cient, under particular conditions, to advance an inter- 
vention, the recent failed attempts by Bayer [13] and 
Amgen [14] to reproduce earlier findings demonstrates 
the dangers with such an approach. 

Finally, we should point out the orphaned study, 83. 
This node has no arrows leading into it, reflecting an 



experiment that is not directly justified on the basis of 
prior evidence within the research program. Although the 
general rule is probably to avoid conducting such stud- 
ies, there may at times be reasons to draw on evidence or 
study designs that are external to the current research pro- 
gram. For example, the experiment may be largely based 
on analogical reasoning, drawing on evidence with a dif- 
ferent drug or a different indication (the phase 3 trials 
of sunitinib and bevacizumab alluded to above could be 
described in just this way). The AERO graph makes this 
design choice explicit - the prudence of which will have to 
be judged by the fruitfulness of the subsequent research. 

Planning the next step 

The final, and perhaps most important, question we want 
to address with the AERO graph is: 'What study should be 
done next?' Given the state of accumulating evidence and 
the current trajectory, what should be the next investiga- 
tion^)? In Figure 3, we have added a single, negative phase 
3 study, 61, following the trajectory from 82. We have also 
added two blue-lined nodes to represent the contemplated 
next steps. 

Herein lies the real power of the AERO graph: In debat- 
ing what study should come next, the visualization can 
sharpen the judgments of researchers, who can now pin- 
point the precise translational trajectory or subset of the 
trajectory that they think ought to inform future research. 



Phase Year 
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Figure 3 Planning future studies. The graph shows twelve 
completed experiments across six years along with two 
contemplated future studies, a fourth phase 2 trial (<5 4 ) and a second 
phase 3 trial {e 2 ). A phase 3 trial (ei ) was initiated following the sole 
positive phase 2 study (52). The result of this phase 3 trial was 
negative, discordant with the earlier phase 2 result. Now researchers 
must decide which study (or studies) to do next: Trust that the 
accumulated evidence is still sufficient to motivate another phase 3 
or return to phase 2 in search of greater consistency and a potential 
explanation for the discordance between 8 2 and €y . 
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For example, let us suppose that e\ was negative for lack 
of clinically significant effectiveness. A researcher could 
argue that despite the negative result, there is still a largely 
positive trend (seven out of twelve studies) across the 
entire trajectory, the drug is very well tolerated (no neg- 
atives at phase 1), and the inconsistency across phase 2 
studies has been instructive in suggesting a novel variation 
on the dosage and schedule that ought to be evaluated in 
a subsequent phase 3 trial Yet, a different researcher may 
look at the same graph, identify a mechanism of action 
common to the inconclusive and negative studies that 
could explain these results, and recommend 84 to test this 
hypothesis in a less costly, phase 2 trial 

The purpose of the AERO graph is thus not to replace 
this kind of critical thinking or eliminate disagreement 
about the state of total evidence. Indeed, researchers may 
even disagree about whether a particular study ought to be 
represented as positive or negative. Rather, the purpose of 
the AERO graph is to sharpen this disagreement by clari- 
fying the state of evidence and translational trajectory. In 
other words, the prudent course of action within a clin- 
ical research program cannot be derived from the state 
of evidence. The decision of what to do next requires a 
negotiation between the practical, pragmatic, ethical, and 
epistemic issues at play. The representational features of 
the AERO model help to make the epistemic aspects of 
this judgment explicit. 

Results 

In its Global Plan to Stop TB 2011-2015, the Stop TB 
Partnership emphasizes the need for improving coordina- 
tion across the entire development and testing trajectory 
[10]. Moxifloxacin, an antibacterial agent in the fluo- 
roquinolone family, is one of the new candidate drugs 
in their pipeline. However, as of 2010, the tuberculo- 
sis research community was confronted with a series of 
inconsistent results across five phase 2 studies with mox- 
ifloxacin, and disagreed about whether the state of total 
evidence supported moving on to conduct phase 3 tri- 
als. The results we present in this section are based 
on our presentation at the Tuberculosis Trials Consor- 
tiums semi-annual meeting in October 2011, where we 
presented an AERO graph of moxifloxacins translational 
trajectory to help broker this dispute. 

Representing the evidence 
Inclusion 

Table 1 summarizes the results of 19 studies evaluat- 
ing moxifloxacin for the treatment of drug-susceptible 
tuberculosis between 1998 and 2009. These studies were 
included based on a hand search through the citations 
of the published phase 2 trial reports. This list was then 
cross-referenced with a PubMed search using the terms 
moxifloxacin and 'tuberculosis;' filtered by clinical trials. 



Table 1 Summary of moxifloxacin-TB studies 



In vitro 


Year 


Key 


Outcome 


Children 


Ji etal. [15] 


1998 


U] 


Positive 


V] 


Gillespie et al. [16] 


1999 


u 2 


Positive 


W] , W2 


Shandil etal. [1 7] 


2007 


u 3 


Positive 


I/V5, Ws 


In vivo 


Ji etal. [15] 


1998 


V] 


Positive 


v 2 


Miyazaki et al. [18] 


1999 


v 2 


Positive 


V3, W] 


Lounisetal. [19] 


2001 


V3 


Positive 


v 4 


Yoshimatsu etal. [20] 


2002 


v 4 


Inconclusive 


V5 


Nuermberger et al. [21] 


2004 


V5 


Inconclusive 


Us, w 3 , 1/1/4 


Phase 1 


Gosling et al. [22] 


2003 


W] 


Positive 


V5 


Pletz etal. [23] 


2004 


Wj 


Positive 


V5 


Gillespie etal. [24] 


2005 


W3 


Negative 


X] 


Johnson et al. [25] 


2006 


w 4 


Positive 


X],X 2 


Nijland et al. [26] 


2007 


Ws 


Negative 




Peloquin et al. [27] 


2008 


w 6 


Inconclusive 


x 4 


Phase 2 


Burman etal. [28] 


2006 


X] 


Negative 


W5, Wq,Xs,X 4 


Rustomjee et al. [29] 


2007 




Positive 


X3 


Conde et al. [30] 


2009 


X3 


Positive 


X5 


Dorman et al. [31] 


2009 


x 4 


Negative 




Wang etal. [32] 


2009 


X5 


Positive 





While this method is less rigorous than a complete sys- 
tematic review, it is sufficient for the illustrative purposes 
of this paper. Further development on harmonizing the 
AERO model with the methods of systematic review is 
underway. 

Extraction 

The year' in Table 1 refers to the year of publication. The 
'key,' generated sequentially by phase, refers to the corre- 
sponding node in Figure 4. The outcome was extracted 
from the published manuscript, based on the direction of 
the observed effect, the authors' recommendation for fur- 
ther research, and any expressed qualifications or reser- 
vations. The studies listed as children are based on a 
transitive reduction of the total network of citations, 
reconstructed to represent the historical translation of 
evidence as accurately as possible. 

Non-positive outcomes 

We acknowledge that the inconclusive status of the two 
in vivo studies is debatable. Despite evidence of efficacy, 
we nevertheless classified Yoshimatsu et al. as inconclu- 
sive due to the authors' concerns about toxicity at the 
recommended dosage [20]. Nuermberger et al. did not 
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Figure 4 Complete AERO graph for moxifloxacin in an anti-tuberculosis regimen. The graph shows 1 9 completed experiments across 1 2 years 
along with four contemplated future studies. The overall trend of study results was positive until the transition into phase 2 (xi ), when significant 
discordance (for example, w 4 -> X] and w 6 -> x 4 ) and inconsistency (that is, negative results in xi ,x 4 vs. positive results in x 2 ,x 3 ,x 5 ) began to 
emerge. Researchers must now decide how to proceed in the face of an equivocal state of total evidence: Investigate mechanisms of discordance 
between animal models and human trials {A); investigate drug interactions (8); further investigate efficacy and evaluate predictivity of specific phase 
2 trial designs (C); or proceed to a decisive phase 3 effectiveness trial (D). 



show an improvement on their primary outcome (time- 
to-culture-negative), but did show a dramatic increase in 
early potency when moxifloxacin was substituted for iso- 
niazid, one of the drugs in the standard regimen [21]. 
Gillespie et al. showed no difference in the early bacte- 
ricidal activity between moxifloxacin and isoniazid [24]. 
Nijland et al. showed that moxifloxacin plasma concen- 
trations are reduced when it is administered with isoni- 
azid and rifampicin [26] . Peloquin et al. showed favorable 
population pharmacokinetics with moxifloxacin, but lev- 
ofloxacin had the most favorable profile in their study [27] . 
Both Burman et al. and Dorman et al. showed no improve- 
ment in the time-to-culture-negative when moxifloxacin 
was substituted into the standard regimen for ethambutol 
and isoniazid, respectively [28,31]. 

Visual representation 

Figure 4 is the complete AERO graph based on the infor- 
mation in Table 1 and the possible future studies, A . . . D. 
It was rendered using the tikz vector illustration pack- 
age for LaTeX. All of the non-straight edges were shaped 
manually in order to aid visual comprehension. The edge 
between u\ and v\ is not an arrow because these two 
studies are published in the same report. 

Analysis 

The first thing to notice about Figure 4 is how much 
messier is the picture of an actual translational research 
program compared to the toy example we discussed 
above. One immediate advantage of the AERO graph is 
that it illuminates the point about translational research: 
It is not a linear process, but a complex network of over- 
lapping investigation types. Phases can even be repeated, 



as when evidence from a downstream phase is used to 
inform a subsequent upstream investigation (for example, 

But however descriptively accurate is this complexity, 
there is a prescriptive question about how organized and 
systematic a research program in translational medicine 
ought to be. Is this overlapping web of studies an intrin- 
sic part of translational medicine? Or can it be made 
more orderly, with one study and one phase proceeding 
after another without the need to backtrack? While we 
will not take up this intriguing question here, it is worth 
pointing out that the AERO model facilitates such an 
inquiry. 

So what can we now say about moxifloxacins transla- 
tional trajectory? Across the entire trajectory, the trend is 
largely positive, with a 3:1 positive to negative ratio. There 
is also evidence of consistency at every phase and no neg- 
ative results in the pre-clinical studies. Although to our 
knowledge there was no explicit evidence threshold estab- 
lished for this trajectory, we can nevertheless observe that 
for both the pre-clinical to clinical and phase 1 to phase 2 
transition, the de facto threshold is three positive studies. 
Thus far, these would all seem to be encouraging prop- 
erties. In fact, prior to the first negative phase 2 study in 
2006, the evidence for moxifloxacin was overwhelmingly 
favorable. 

After 2006, significant inconsistency and discordance 
emerges. Two of the five phase 2 studies, x\ and #4, are 
negative and show discordance with the earlier phase 1 
and pre-clinical outcomes. Indeed, there is a subset of the 
total trajectory, proceeding through w% to %\ and then W5 
and #4, that is both consistently and concordantly nega- 
tive. This underscores the fact that robustness is not only 
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a property of positive results. Results can also be robustly 
negative. 

Yet, one negative sub- trajectory is not necessarily fatal 
for the whole translational trajectory. In this case, we can 
see an overall trend toward positive outcomes, but it is 
not obvious that this trend is sufficient to justify a phase 
3 trial. Therefore, we now consider how the AERO graph 
can sharpen arguments for or against each of the four 
proposed future studies. 

A: The argument in favor of A, another in vivo study, 
is supported by the striking discordance between the 
pre-clinical and clinical results along the following sub- 
trajectory: 

• • • V5 W3 X\ X4 A 

Given that the in vivo models and phase 2 trials are both 
evaluating an efficacy endpoint, rather than the safety, 
pharmacokinetic, or early activity endpoints evaluated in 
phase 1, the discordance between in vivo and phase 2, 
combined with phase 2 inconsistency, casts a reasonable 
doubt on the predictive power of the animal models. For 
example, Dorman et al.s (#4) hypothesis, comparing the 
replacement of moxifloxacin for isoniazid in the standard 
regimen, was informed by Nuermberger et al.s (V5) find- 
ing in a murine model. The failure to translate this result 
supports revisiting the in vivo design and re-evaluating the 
import of in vivo experiments. 

But we should note that pursuing another in vivo study 
does not necessarily invalidate the prior phase 1 or phase 2 
results. Those studies can still be internally valid and their 
evidence can remain reliable and informative in future 
analyses. The rationale in favor of another animal study 
is that it may be more productive or efficient in the long 
term to try and explain the discordance between some 
of the extant animal and human results, rather than sim- 
ply continuing to test moxifloxacin-containing regimens 
in humans. 

B: The argument in favor of B, another phase 1 study, 
is supported by the inconsistencies at phases 1 and 2. In 
addition to the above sub -trajectory, which tracks the neg- 
ative results at phase 2, this line of reasoning adds a second 
trajectory of emphasis: 

. . . V5 — »> W3 —> %\ —> #4 =>> B 
. . . x\ W5 B 

These sub-trajectories suggest a need to better under- 
stand the pharmacokinetic properties of moxifloxacin 
when used with the other drugs in the standard anti- 
tuberculosis regimen. Nijland et al.s (ws) finding that 
some combinations with moxifloxacin produce lower 
plasma concentrations relate directly to this point. 



C: The argument in favor of C, another phase 2 trial, 
emphasizes differences in experimental design across 
the phase 2 studies, supported by the following sub- 
trajectories: 

. . . v 5 11/4 x\ x 3 -> X5 C 
. . . x\ —> X4 C 

. . . X2 —> X3 —> Xs C 

Just as we discussed a robustly negative sub- trajectory, 
a researcher could emphasize the robustly positive sub- 
trajectory and claim that if any doubts about moxi- 
floxacins potential efficacy remain, that these ought to be 
eliminated by another, rigorous phase 2. 

Importantly, the justification for C turns, in no small 
part, on assumptions about the purpose of the phase 2 
trial and the threshold of evidence for proceeding to phase 
3. If phase 2 trials are supposed to limit the number of 
candidate interventions for phase 3 strictly, then it may 
be reasonable to require at least a 2:1 positive-to-negative 
trial ratio, for example, before advancing. In which case, 
C seems a reasonable option. If, however, phase 2 trials 
are only supposed to rule out dangerous candidates, then 
three positive trials is arguably sufficient, and therefore, C 
would seem to be unnecessary. 

D: This leads to the argument for D, a phase 3 trial, which 
could be justified on the grounds that moxifloxacin has 
been evaluated in three positive phase 2 trials, has not 
been ruled out as a novel candidate for inclusion in an 
effective anti-tuberculosis regimen, and is supported by 
the sub- trajectory: 

. . . V5 W4 -> X2 ~> X3 X5 D 

As we have already mentioned, the overall picture of mox- 
ifloxacins trajectory is largely positive - not perfect, but 
arguably robust. Therefore, it may be reasonable to pro- 
ceed to phase 3, despite the inconsistency across phase 
2. Indeed, we should acknowledge here the REMox-TB 
trial (NCT00864383), an in-progress phase 3 trial evaluat- 
ing a moxifloxacin-containing regimen for the treatment 
of drug-susceptible tuberculosis. The results of this trial, 
although not yet available, can be understood, in part, 
as an empirical test of this line of reasoning. A positive 
and reproducible finding in the REMox-TB trial would 
be evidence that the AERO graph of Figure 4 reflects a 
promising trajectory. Inversely, a negative finding should 
cast doubt on the idea that such a trajectory is robust 
enough to justify a phase 3 trial. 

Summary 

Whichever direction is ultimately selected for future 
research, the AERO model can be dynamically updated in 
light of those new study results as they become available. 
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For example, suppose that moxifloxacin researchers were 
to pursue both options A and C - another animal study 
and another phase 2 trial, respectively. And assume fur- 
ther that both of these turn out to be positive. This could 
potentially show that the researchers now better under- 
stand the predictive relationship between the animal and 
human models and perhaps finally tip the balance of phase 

2 evidence clearly in favor of a moxifloxacin regimen. 
Yet, even before these studies are executed, researchers 

could use the AERO graph to strategize about future states 
of evidence. In contemplating option B, another phase 1 
trial, one might reasonably question if that result, what- 
ever it turns out to be, would be informative and useful 
enough to justify the opportunity cost of not pursuing A, 
Cor D. 

We should also acknowledge that these alternative 
research strategies are neither exhaustive nor mutually 
exclusive. A well-funded research program may be able 
to pursue all of these and other options simultaneously. 
The aim in this section is simply to show how the AERO 
graph can help to clarify and sharpen the rationale for the 
various possibilities and in so doing, aid decision-making 
about the directions for further research. 

That said, the more fundamental questions elucidated 
by the moxifloxacin trajectory - calling into question the 
predictive value of the animal model, reconsidering the 
purpose for phase 2, and the appropriate thresholds of 
robustness at each phase transition - should not be over- 
looked, even if the decision is made to proceed to a phase 

3 trial. Part of what Figure 4 illustrates is that there are 
many unanswered questions about moxifloxacins trajec- 
tory and the underlying causal relationships, and until at 
least some of these are addressed, a phase 3 trial, whatever 
its outcome, will be less informative that it could be, and 
hence, reflect an inefficient research strategy over the long 
term. 

Discussion 

Stepping back from this specific case study, there are a 
number of possible elaborations and applications of the 
AERO modeling approach that are worth discussing. To 
begin with, the color-coding we presented here, which 
classifies studies as positive, negative, or inconclusive, has 
significant virtues of simplicity and ease of interpretation, 
but this is far from the only option. For example, a contin- 
uous shading scale could be used to represent the effect 
size or precision of each study; or a five-point ordinal scale 
could be used, corresponding to the region of posterior 
interval. The fundamental structure of the AERO model 
is the directed acyclic graph with study-type strata; the 
other graphical properties can be extensible in any way 
that supports decision-making. 

The AERO graph can also be thought of as reflect- 
ing the maturity of causal knowledge within a given 



research domain. A robust field of mostly green nodes 
would suggest that investigators have full command of the 
mechanisms in the causal system, whereas a field of pre- 
dominantly red or yellow nodes would suggest utter lack 
of contact with the causal factors. A pattern of pre-clinical 
green that consistently turns to red in clinical translation 
should cast doubt on the validity of pre-clinical models. 
A thin thread of green would perhaps suggest just a lucky 
find, while an evenly balanced network of green and red 
nodes would suggest a new, emerging domain with which 
investigators may have only a limited understanding. 

This relates to another extension of the approach - com- 
paring and contrasting multiple trajectories. We could 
see such a comparative analysis being useful in a number 
of ways: For example, the AERO graph of a successfully 
translated agent could be used as the model for how new 
agents in a treatment domain ought to be vetted; or a 
population of AERO graphs could elucidate systematic 
differences in efficiency or risk and benefit across different 
research domains. 

Looking ahead to envision the potential utility of the 
AERO model, particularly as a means to improve the effi- 
ciency and coordination of the drug development enter- 
prise, we believe that publications and protocols could 
both benefit from including an AERO graph. Just like 
other kinds of visual tools, such as Thorpe et al.s PRE- 
CIS graph [33] or Langan et al.s graphical augmentation to 
the meta-analytic funnel plot [34], serve to aid judgments 
about the quality and direction of evidence, so too can the 
AERO model help to make invisible or inchoate properties 
of the research system explicit. 

Indeed, AERO graphs could provide a convenient check 
for investigators, institutional review boards, journal edi- 
tors, and physicians. The comprehensibility of the repre- 
sentation allows for anyone even modestly familiar with 
a particular domain to be able to compare and contrast 
alternative representations. As a consequence, a report 
that ignores a substantial portion of the available evi- 
dence can be more easily detected, since its represen- 
tation will differ dramatically. This is not to contradict 
what we claimed earlier about disagreement, as different 
researchers may have equally legitimate interpretations of 
their field. However, the apparent contrast between inter- 
pretations of the evidence will demand an explanation for 
why the authors represented the field in one way rather 
than another. 

Conclusion 

The AERO model provides an immediate visual repre- 
sentation of the number of studies done at any mode, 
depicting both the direction of evidence and the relation- 
ship of each study to the larger translational trajectory. 
In so doing, it helps to address the widespread con- 
cerns about efficiency, coordination, and organization in 
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translational medicine. To be sure, the visual represen- 
tation does not capture all of the available information 
about a research program. For example, the details of dif- 
fering experimental designs or conduct may be relevant to 
understanding failures of robustness. Just as we showed 
in the previous section, the argument for one course of 
action over another relies upon knowledge of these addi- 
tional details about each experiment. Nevertheless, the 
AERO model provides a systematic representation that 
is capable of sharpening these judgments and reveal- 
ing some of the existing patterns across a translational 
trajectory. 

We recognize that the approach we have presented here 
is but a preliminary sketch. While we have applied the 
AERO model to a single case study, much more work is 
needed to develop the approach to its full potential. But 
particularly given the stakes, thinking about ways to better 
analyze and judge the structure of clinical research pro- 
grams as a whole seems a vital line of inquiry. The AERO 
model is one piece of this inquiry. 
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